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ABSTRACT: There is rising demand for automated digital twin construction based on point cloud scans, 
especially in the domain of industrial facilities. Yet, current automation approaches focus almost exclusively on 
geometric modelling. The output of these methods is a disjoint cluster of individual elements, while element 
relationships are ignored. This research demonstrates the feasibility of adopting Graph Neural Networks (GNN) 
for automated detection of connectivity relationships between elements in industrial facility scans. We propose a 
novel method which represents elements and relationships as graph nodes and edges respectively. Element 
geometry is encoded into graph node features. This allows relationship inference to be modelled as a graph link 
prediction task. We thereby demonstrate that connectivity relationships can be learned from existing design files, 
without requiring domain specific, hand-coded rules, or manual annotations. Preliminary results show that our 
method performs successfully on a synthetic point cloud testset generated from design files with a 0.64 F1 score. 
We further demonstrate that the method adapts to occluded real-world scans. The method can be further extended 
with the introduction of more descriptive node features. Additionally, we present tools for relationship annotation 
and visualisation to aid relationship detection. 
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1. INTRODUCTION 


Ageing industrial facilities often lack essential documentation, resulting in sub-optimal maintenance and 
breakdowns. Digital twins remedy this and assist in the operation and maintenance of industrial facilities. 
However, generating twins for existing facilities is a laborious and time-intensive process that outweighs the 
perceived benefits offered by the twins. (Agapaki et al., 2018). While this has resulted in significant interest in 
automation, current approaches merely segment elements and model their geometry. However, industrial facilities 
are composed of a vast number of interconnected elements of various categories; thus, the identification of their 
connectivity relationships is a crucial, yet challenging step in the digitisation process. 


The prevalent methodology for constructing geometric DTs from existing facilities, known as “Scan-to-BIM” 
(Tang et al., 2010) consists of the following steps: (1) raw data collection, (2) data preparation, (3) geometric 
modelling, and (4) semantic enrichment of the model. The focus of this paper is the final step. 


Semantic enrichment’ refers to the incorporation of various forms of additional information into a digital twin to 
enhance its value. Some common examples are element relationships, material information, damages to elements, 
and code compliance information. There are various types of ‘element relationships’ within industrial facilities. 
Tang et al (Tang et al., 2010) identifies three commonly found relationship types. Namely, topological 
relationships (e.g., a pipe being connected to an elbow), aggregation relationships (e.g., a pipe being within the 
HVAC (Heating, Ventilation and Air Conditioning) system) and containment relationships (e.g., a window 
belonging to a wall). This paper focuses on topological relationships. 


Topological relationships between various elements of a facility are a key component of its documentation. For 
instance, when diagnosing faults within building systems (Tang et al., 2010), carrying out maintenance tasks, or 
checking for code compliance (Bloch & Sacks, 2020), topological relationships must be identified in advance. 
They are also crucial when analysing sub-systems within a plant, which assists maintenance and monitoring. 


A variety of industry tools such as Trimble RealWorks, Leica Cyclone, ClearEdge3D EdgeWise are used in the 
DT construction process. Currently, element relationships modelling between elements is largely a manual process 
with industry tools providing limited automation. Tools such as EdgeWise can connect adjacent pipes, but not 
other piping elements. AVEVA E3D and PointSense offer the ability to derive pipe branches but require user 
guidance through point picking. A feature comparison of popular tools is given in table 1 (Son et al., 2015). The 
requirement of manual guidance throughout the modelling process is one of the primary pitfalls of current software 
solutions, necessitating significant expert labour. 
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Table 1. Comparison of automation features and pipe modelling functions (SA=Semi-automated, FA=Fully- 
automated, NA=Not-available, Source: (Son et al., 2015) 


Task Trimble RealWorks Leica Cyclone ClearEdge3D EdgeWise 

Automation features 

Pipe Detection SA FA FA 

Part Recognition (e.g., elbows, tees) FA NA FA 

Model Creation FA FA FA 

Pipe modelling functions 

Straight pipe FA FA FA 

Elbow NA NA FA 

Tees NA NA FA 


This paper proposes a novel method for automatically identifying topological relationships between elements 
within industrial facility models using GNNs. We demonstrate our model’s performance both on synthetic and 
real-word data. Furthermore, we present tools for relationship visualisation and annotation to aid in the 
relationship detection process. Crucially, this research demonstrates the applicability of graph inference for 
element relationship inference in BIM. 


2. BACKGROUND 


Current literature on industrial facility DT construction predominantly encompasses the first three steps of the 
Scan-to-BIM process. Annotated point cloud datasets such as CLOI contains elements from classes such as 
Channels, Valves, I-beams, Flanges, Elbows, Cylinders and Angles (Agapaki et al., 2019). A prominent instance 
segmentation method utilizes CLOI-NET, a modified PointNet++ based neural network to identify element point 
clusters using the above dataset. This achieves 73.2% mean precision and 71.1% mean recall over all classes. 
However, results for more complex shapes such as flanges are considerably lower, especially in the presence of 
occlusions. Another approach proposes ResPoint++, which uses an encoder-decoder structure trained on I-Beams, 
R-Beams, pumps, pipes, and tanks (Yin et al., 2021). Xie et al proposes PipeNet for modelling straight pipes via 
centreline prediction (Xie et al., 2023). Once element point clusters are retrieved, they are geometrically modelled 
with approaches such as CAD model matching (Agapaki & Brilakis, 2022). However, the final semantic 
segmentation step, particularly in the form of relationship inference is yet unsolved, in industrial facilities as well 
as in other domains. Moreover, scan datasets with annotated relationships currently do not exist. 


Traditionally, element relationships are defined with various data schemas such as IFC. However, graph 
representations of IFC models have recently become prominent due to their ability to query information more 
effectively. In a graph representation, each element is represented by a graph node. Relationships between 
elements are depicted by edges. Graphs are well suited for the representation of building information as both 
spatial and non-spatial information can be stored as node or edge properties within a graph (Ismail et al., 2018). 


Previous attempts at automated detection of element relationships rely on hard coded rules. Nguyen et al. proposes 
an algorithmic approach to inferring relationships such as adjacency, containment, intersection, and connectivity 
from CAD models of elements (Nguyen et al., 2005). This requires complete CAD models without occlusions 
and is constrained by many assumptions. Another approach infers connecting tees and elbows based on pipe 
centrelines to predict pipelines (Oh & Kwang, 2021). Hard coded rules are unique to their domain. Thus, these 
approaches cannot scale to various domains, and are limited to a few common scenarios. To our knowledge, no 
published work exists that attempts to derive relationship information between various elements a laser scan. 


Such a task would require an understanding of the nature of element relationships within an environment. For 
instance, the existence of a pipe and an elbow in proximity and in alignment suggests that the two elements are 
linked. There is a diverse and non-exhaustive set of such instances where relationships can be inferred, especially 
in the presence of occlusions and barriers such as walls. Furthermore, these vary between domains; the types of 
relationships in a bridge are vastly different from those in an industrial facility. Thus, rule-based approaches to 
relationship inference tend to perform poorly. We posit that a method of automatically learning the nature of 
relationships in a particular domain is better suited for this use case. In particular, we focus on GNNs, which are 
geometric deep learning approaches capable of learning directly from graphs. 


GNN architectures can be split into spatial and spectral GNNs. Spatial GNNs such as GraphSAGE (Hamilton et 


al., 2017) and Graph attention Networks (GAN) (Veličković et al., 2017) create vector embeddings of graph nodes 
and aggregate features of adjacent nodes. In contrast, spectral GNNs such as Graph Convolution Networks (GCN) 
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(Kipf & Welling, 2016) are based on graph message passing. Other influencing factors include size of the graph 
and type of task. Tasks can be broadly categorized into three types: node classification, link prediction and graph 
classification. The choice of architecture for a particular task is influenced by a variety of factors. For instance, 
GraphSAGE and GCN are by default suited for graphs without edge features. Furthermore, they both behave as 
inductive frameworks, allowing them to scale to nodes that are unseen during the training process. However, 
inductive frameworks are unable to perform prediction on a completely edgeless new node, as information cannot 
be propagated in the absence of edges. Methods such as Edgeless-GNN (Shin et al., 2021) address this 
shortcoming by introducing pseudo edges based on similarities in node features. These edges ensure message 
propagation to new nodes. 


There are very few applications of GNNs in the buildings domain. Buruzs et al. and Wang et al. both utilize graph 
representations of IFC models for the task of room type classification. They utilize a GCN and a modified version 
of GraphSAGE architecture respectively (Buruzs et al., 2022; Wang et al., 2022). They both model the task as a 
node classification problem and focus on indoor living spaces. Some of the edge features used within the graph 
representation include type of connection (e.g., Door vs wall) and material of connection (wooden vs metal door), 
while some of the node features include volume, height, oriented bounding box dimensions etc. The above 
methods demonstrate the suitability of graph learning in BIM. 


The above findings demonstrate that we do not yet know how to automatically detect relationships between 
industrial facility elements. We merely know how to identify individual elements in isolation, but a digital twin 
should represent the connectivity and interactions of the system. The aim of this work is to address this gap in 
knowledge by answering the research questions; (a) Which strategy to utilize for automated inference of 
topological element relationships with high precision and recall? And (b) How to train an element relationship 
inference model in the absence of annotated relationship datasets? 


3. PROPOSED SOLUTION 


We propose a method for automated topological relationship detection between elements. The scope of this 
research is limited to cylinders, elbows, tees, and flanges. These elements account for a majority of the modelling 
workload (Agapaki et al., 2018). 


Segmented element point clusters of existing industrial facilities are the input data source. These individual point 
clusters are extracted from a scan using an existing instance segmentation method such as CLOI-NET. Elements 
and their relationships are represented in the form of a graph. Each element is modelled as a graph node, and its 
geometric features and element class are encoded into a graph node feature vector. Relationships are represented 
by edges. Relationship detection is modelled as an edge prediction task and a GNN is trained for this purpose. 
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Figure 1. Overview of the proposed solution 


Element geometry information is encoded into graph node features. These features are used by the GNN to learn 
the influence of element geometry on connectivity between elements. A more descriptive node feature provides 
additional information to the GNN. Possible features include: (a) minimum oriented bounding boxes (MOBB) of 
elements, (b) element-specific parameters such as axis, radius etc., (c) sampled subset of points, and (d) learned 
feature vectors. Oriented bounding boxes can be easily derived but are limited to crude information regarding 
element geometry and position. These are heavily affected by outliers and errors in instance segmentation. In 
contrast, element specific features are more difficult to extract from point clouds, but are more descriptive, 
especially for shapes such as elbows. They can represent geometries accurately with few parameters. Examples 
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include element position (e.g., centre point), orientation (e.g., cylinder axis) and element geometry (e.g., radius, 
length). Element parameters of simpler shapes such as cylinders may be extracted using methods such as 
RANSAC, but more complex shapes such as elbows can prove challenging. Yet another approach would be to 
encode the element point cluster into a feature vector. Such an encoding may be generated automatically by using 
a deep learning approach such as PointNet (Qi et al., 2016) and would theoretically be capable of robustly 
representing the element. 


Manual annotation of a relationship dataset to train the GNN would require a significant amount of time and 
labour. Thus, we generate training data from design files which contain element relationship information. In terms 
of GNN architecture, we opt for GraphSAGE, as it is (a) capable of inductive learning, (b) suited for link 
prediction, (c) scales linearly in the number of graph edges and (d) has been successfully utilised for previous 
BIM applications (Wang et al. 2022). 


4. RESEARCH METHODOLOGY 


This research is designed upon the assumption that topological relationships between elements can be inferred 
from their geometric features, and that the nature of such relationships can be learned by a neural network. We 
further assume that the data loss from compressing point cloud instances to node features does not significantly 
affect the ability of a GNN to identify element relationships. 


The dataset used for training is composed of design files for an offshore Liquid Natural Gas hub in NavisWorks 
format. The subset utilized for experimentation comprises of two sub sections of the site containing around 37,000 
elements with around 31,000 unique topological relationships (Figure 2). Element relationships were extracted 
using a python script through the NavisPythonShell plug-in and geometries were extracted by the NavisTools 
plug-in. 
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Figure 2. A subsection of the input dataset (left) and element relationship frequencies (right). 


Out of the proposed node feature representations, we test the bounding box and sampled subset of points strategies. 
The relative simplicity of these methods provides a starting point for assessing the feasibility of our solution. For 
the bounding box representation, we calculate the principal axis vector, centre-point, and dimensions of the 
minimum oriented bounding box. For the latter representation, we randomly sample 100, 500, and 1000 points 
for each element (Figure 3). 
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Figure 3. Node features derived from bounding box (left) and Element points sampled at 1000, 500, and 100 
points (right) 
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We experimentally determine that node features derived from bounding box geometry provide best results. Our 
graph representation consists of a graph whose nodes represent individual elements, and whose edges denote 
topological relationships between the elements. Each node is represented by a feature vector consisting of the 
above bounding box parameters. Specifically, the principal axis is represented by a 3D unit vector, and centre 
point and dimensions are also represented by 3D vectors. The class label of the element is appended to the feature 
vector using a one-hot encoding. 


The link prediction GNN and graph dataset were implemented using PyTorch and Deep Graph Library. The 
training/validation set consisted of around 20,000 elements and 17,000 topological relationships. 10% of this set 
was reserved for validation. Furthermore, a separate, disjoint section of the design file dataset was used as a test- 
set, ensuring that the trained model scales to unseen graphs. The test-set comprised of around 17,000 elements 
and 15,000 relationships. For training, we generate a positive graph containing all edges in the training set, and a 
negative graph containing the inverse of those edges. As each element could potentially be connected to every 
other element, the negative graph contains an exponentially large number of edges. Therefore, we sample a subset 
of edges to create a balanced training set. The sampling is performed dynamically during training to include all 
potential negative edges. Furthermore, we restrict our search to potential edges between elements within a pre- 
defined range and generate pseudo edges based on physical proximity between elements. 


Our proposed GNN is based on the GraphSAGE architecture and contains 2 GraphSAGE layers for node feature 
aggregation, as well as a 2-layer Multi-Layer Perceptron (MLP) for edge feature computation. The node features 
act as input for the 1 layer of the GNN. Ina single layer, features of each nodes’ neighbours are aggregated using 
mean aggregation, and combined with its previous features and a trainable weight vector. Next, edge features are 
computed to calculate probability of a link. We evaluate two prominent edge feature predictors, namely dot 
product and MLP, and determine that an MLP with two layers and a Rectified Linear Unit (ReLU) activation 
function yields best results. The model is trained using Binary Cross Entropy loss. 


5. RESULTS AND DISCUSSION 


Performance comparisons of hyperparameters are given in table 2. Training losses failed to converge when 
training with sampled points as node features. Thus, all results listed in the table utilize bounding box parameters 
and element class label as node features. All models were trained for 500 epochs, with Adam optimizer and a 
learning rate of 0.01. 


Overall, the system achieves a recall of 0.88, precision of 0.51 and F1 score of 0.64 on the test-set at 0.5 
classification threshold. A breakdown of model performance on relationships between various element types is 
given in table 3. The drop in both precision and recall is primarily due to errors in pipe-pipe relationships, 
stemming from disjoint piping elements. 


Table 2. Performance comparisons between various hyperparameters on validation set 


Precision Recall F1 score AUC-ROC 

Node representation update method (with MLP — 2 layers) 

1 SageGRAPH layer 0.967 0.771 0.858 0.879 

2 SageGRAPH layers 0.980 0.738 0.842 0.967 

3 SageGRAPH layers 0.976 0.737 0.840 0.924 
Edge feature computation method (with SageGRAPH — 2 layers) 

Dot product 0.700 0.955 0.806 0.900 

1 MLP layer 0.486 0.677 0.566 0.451 

3 MLP layers 0.937 0.736 0.825 0.855 


We also test model adaptability to real world data by testing on a manually annotated subset of the CLOI dataset 
containing flanges, elbows and pipes. Tees were excluded due to their low prevalence. The dataset contains around 
600 relationships between around 1100 elements. We develop a new relationship annotation and visualisation tool 
based on LabelCloud, an open-source point cloud bounding box annotation tool (Sager et al., 2021) for this 
purpose. The model achieves a recall of 0.99, precision of 0.75 and F1 score of 0.85 on this dataset. The higher 
performance is a result of the dataset being less complex than the design dataset with less densely packed elements. 


891 


CONVR 2023. PROCEEDINGS OF THE 23°° INTERNATIONAL CONFERENCE ON CONSTRUCTION APPLICATIONS OF VIRTUAL REALITY 


Table 3. Recall and precision by element type, in design file (top) and CLOI (bottom) testsets 


Precision Flange Elbow __ Tee Pipe Recall ___ Flange _ Elbow _ Tee Pipe 
Flange 0.74 0.94 0.88 0.93 Flange 1.0 0.98 1.0 0.97 
Elbow 0.89 0.60 0.96 Elbow 0.97 0.97 0.94 
Tee 0.94 0.71 Tee 1.0 0.94 
Pipe 0.24 Pipe 0.76 
Precision (CLOI) Flange Elbow _ Pipe Recall (CLOI Flange Elbow Pipe 
Flange 0.25 0.93 0.85 Flange 1.0 1.0 1.0 

Elbow 0.14 0.92 Elbow 1.0 0.98 
Pipe 0.70 Pipe 1.0 


Additionally, we visualise results by programmatically generating cylindrical elements at connection points to 
denote topographical relationships (Figure 4). Most false positives and false negatives occur on elements with 
smaller radii. In particular, the model predicts false edges on small parallel pipes. Parallel pipes connected via two 
elbows are another cause of false positives. The model is also more likely to miss relationships in densely packed 
regions. Notably, errors are more prevalent among smaller connecting elements such as tees. This may be 
attributed to shortcomings of the node feature representation. Unlike pipes, smaller elements such as tees or 
elements do not have an unambiguous principal axis. Therefore, a bounding box representation may be inadequate 
to represent their geometry. Switching to more descriptive features such as element parameters may prove to be 
a valuable avenue for future work. Visual analysis of CLOI dataset results (figure 5) demonstrates that errors are 
primarily false positives mainly caused by noisy point clusters. This is explained by the high sensitivity of the 
bounding box representation to noisy points. 


Figure 4. Model predictions (left), and failure cases (right) on design file testset and predictions on CLOI dataset 
(bottom) (True positive=Green, False positive=Yellow, False negative=Red) 
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Figure 5. predictions on CLOI dataset. Edges are denoted by lines between edges of element bounding boxes 
(True positive=Green, False positive=Yellow, False negative=Red) 


6. CONCLUSION 


We propose a novel method for automatically identifying topological relationships between elements within 
industrial facility models using GNNs. Specifically, this research is the first to accomplish this task without a rule- 
based approach. While significant improvements are required to match the precision and recall of human 
annotators, our method demonstrates the feasibility of automated relationship inference, and can be used as 
guidance for annotators. Many failure cases are caused by limitations of the bounding box node representation. 
These include sensitivity to noisy points and inability to represent geometry of smaller elements accurately in 
densely packed areas. Thus, there is significant potential for future improvement by substituting a more advanced 
representations such as element parameters or a learned feature representation. Another limitation of the proposed 
method is low performance in the presence of occluded points, which are common in large scale indoor scans. 


The method can also be extended to detection of aggregation relationships such as facility subsystems. 
Furthermore, the inferred relationship information may also be utilised as additional context to improve instance 
segmentation performance. Crucially, in contrast with previous hand-coded approaches in various domains, this 
paper presents an automated alternative to relationship inference, which is crucial to the semantic enrichment step 
of the scan-to-BIM process. It is thus suited for more complex scenarios, and is easily adaptable to other domains, 
making digital twinning more accessible to previously unexplored infrastructure domains. 
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